Lecture 3: Continuous times Markov Chains. Poisson Process. Birth and Death Process. 1 Continuous Time Markov Chains

In this lecture we will discuss Markov Chains in continuous time. Continuous time Markov Chains are used to represent population growth, epidemics, queueing models, reliability of mechanical systems, etc. In Continuous time Markov Process, the time is perturbed by exponentially distributed holding times in each state while the succession of states visited still follows a discrete time Markov chain. Given that the process is in state i, the holding time in that state will be exponentially distributed with some parameter λi, where i can represent the current population size, the number of alleles A1 in the population, etc. These holding times basically control how rapidly the movements (changes of states) of the chain take place. Additionally, given the knowledge of visited states, the holding times are independent random variables. For a Continuous Markov Chain, the transition probability function for t > 0 can be described as